Deep Reinforcement Learning for Pairs Trading

Reinforcement learning (RL) [1] differs from traditional supervised machine learning in the sense that it not only considers short-term consequences of actions/decisions, but also long-term outcomes. Because of recent advances in deep learning, model-free deep reinforcement learning (DRL) has proven successful in various applications, as with the success of a deep Q-network (DQN) in the Atari game [2]. A common application of RL is stock trading, as the ultimate goal is to make long-term profit while accounting for the fact that current profits are valued more than future ones. We applied DRL in stock markets to train a pairs trading agent with the goal of maximizing long-term income, albeit possibly at the expense of short-term gain. Briefly, pairs trading is an investment strategy that analyzes pairs of stocks that have a common trend and involves making investment decisions when the stock prices diverge, on the assumption that they will converge back to the trend in the near future [3]. The motivation for our work is clear, as a wellperforming investment model will be very lucrative for both corporate and individual investors.